AI models can generate entire features in minutes, but is the code correct? We built Android Bench, our AI evaluation benchmark, to answer this question after analyzing thousands of generated code changes.
This talk shares insights from our journey, including all the ways we saw models cheats, how prompt specificity leads to higher code quality and how different agents approach the same task.
Resources:
Android Bench 2.0 →
Speakers: Tomáš Mlynarič
Chapters:
0:00 - Introduction to Android Bench
2:40 - Introducing Android Bench 2.0
5:05 - Grading challenge
9:34 - Cheating challenge
12:29 - Insights from benchmark results
18:27 - How models test their own work
21:42 - Pass rates versus completion rates
23:48 - Benchmark comparison and wrap-up
Watch the tools & performance sessions from Android Dev Summit 2026 →
Watch all the sessions from Android Dev Summit 2026 →
Subscribe to Android Developers →
#AndroidDevSummit2026
Event: Android Dev Summit 2026
Products Mentioned: Android Bench, Developer Tools
|
2026/10/30
youtube
